1. Field of the Invention
The present invention relates to an information embedding method and apparatus for embedding information in original data such as image data by using a deep layer encryption method or the like and to a supplementary information reading method and apparatus for obtaining supplementary information related to a the original data based on the embedded information, and also to a computer-readable recording medium storing a program to cause a computer to execute the above methods.
The present invention also relates to an image processing method and apparatus, and more specifically, to a method and apparatus for processing the original data wherein the supplementary information related to the data has been embedded by using deep layer encryption.
2. Description of the Related Art
Various kinds of information such as image, audio, and movie data have been digitized, and data in various file formats exist, depending on a recording medium to store the data or on application software to use the data. For example, as formats for image data used on the Internet, JPEG, GIF, and FlashPix proposed by Eastman Kodak Co. have been known, and image data in a format suitable for the content of an image are exchanged. Following installation of infrastructure of such open networks, the chances of processing or using data via various kinds of recording media and applications have been growing. For example, the chances of repeatedly processing image data by using different applications or saving image data in various formats have been growing.
For this reason, it has been very useful to add information to data, in prospect of more complex and general usage of the data in an open network environment. For example, when image data obtained by a digital camera or the like are converted to output data such as a hard copy, ways in which the image data are manipulated are different depending on a photographed scene. In a case like this, if light source information of the photographed scene, such as color fog, under-exposure, over-exposure and the like, is known, appropriate image processing can be carried out on the image data, depending on the light source information. Furthermore, by adding a keyword to original data, various useful effects are expected, such as capability of a search for the original data after database compilation. Meanwhile, for print data used in the field of printing, information regarding process management such as an instruction of trimming or correction of an image and information as to which page the image belongs to is added to image data for publishing, and process management can be carried out by compiling the management information into a database.
How to add the supplementary information to original data is an issue to be considered. In reality, supplementary information is attached to an original data file as a file separate from the original data file, or written in a header of the original data file. However, in the former case, if various kinds of processing are carried out on the image data by various kinds of applications, or if format conversion is carried out, correspondence between the original data and the supplementary information may not be maintained. In the latter case, the supplementary information in the header cannot be read in some cases depending on application software, or the information written in the header may be lost as a result of carrying out format conversion after reading the data. In other words, the file header has a basic part and an extension part, and the supplementary information is included in the extension part which may be lost or altered by repeated writing or reading.
Meanwhile, various methods for embedding authentication information or copyright information in original data by using deep layer encryption have been proposed (see Japanese Unexamined Patent Publication Nos. 8(1996)-289159, 10(1998)-108180, 9(1997)-214636, for example). According to these methods, when the original data are image data for example, by embedding authentication information and copyright information in a redundant portion of the data, the information embedded in the data can be read and displayed by using an apparatus and software for reading the information, although the embedded information cannot be confirmed by simply reproducing the image data. This kind of deep layer encryption is described in detail in various references (for example, see K. Matsui, xe2x80x9cDigital Watermarkxe2x80x9d, O Plus E No. 213, 1997).
As methods for this kind of deep layer encryption, methods of various types such as a pixel space utilizing type, a quantization error utilizing type, and a frequency range utilizing type have been known. The pixel space utilizing type takes out a plane of 3xc3x973 pixels around a target pixel for example, and supplementary information is embedded in the surrounding 8 bits. The quantization error utilizing type pays attention to a quantization error which occurs in a process of compressing image data. Quantization output is regulated into even numbers and odd numbers by using 0 and 1 of a bit series of the supplementary information and the supplementary information is embedded in the image data as apparent quantization noise. xe2x80x9cVideo Steganographyxe2x80x9d (in Japanese) by K. Matsui published from Morikita Shuppan in 1993 has the details of the quantization error utilizing type. The frequency range utilizing type is a method of embedding supplementary information in a frequency range of an image area to which the human visual system is comparatively insensitive. For example, since a high frequency component in an image is the range wherein the human visual system is comparatively insensitive, supplementary information can be embedded by decomposing image data into a plurality of frequency bands so that the information can be embedded in the high frequency band and by reconstructing the image data thereafter. As a characteristic of the human visual system, sensitivity to color difference and to chroma information is generally lower than to luminance information. Therefore, a range wherein invisible recording is possible exists in a difference between the luminance and the color difference or between the luminance and the chroma information. Consequently, the supplementary information can be embedded in this range.
In addition to these methods, methods of embedding supplementary information in a bit plane having a low S/N ratio as an image by concealing the information in noise redundancy, or in redundancy of an information change in a pixel block (space) of a predetermined range, or in a quantization error occurring in the case of degeneracy of data amount due to coding for data compression, can be used, for example.
An information embedding method using information conversion redundancy of a pixel block in a density pattern for representing tones in a binary image will be explained next.
A density pattern method is a method of representing multi-valued tones by a binary bit map. FIG. 16 is a diagram showing the case of representing tones by an area change of a unit, using 4 binary pixels as one unit. The number of tones Leq which can be represented by the specific number of bits L and nxc3x97n pixels can be expressed as follows:
Leq=(Lxe2x88x921)n2+1xe2x80x83xe2x80x83(1)
Therefore, for the case shown in FIG. 16, 5 tones can be expressed. As is obvious from FIG. 16, as a pattern of showing one quantization level (the same tone), several patterns exist depending on which of the 4 pixels has a bit representing black. For example, there is only one pattern for the quantization levels 0 and 4. However, the quantization levels 1 and 3 have 4 patterns each and the quantization level 2 has 6 patterns. In other words, since the number of patterns is determined by a combination arrangement of m black pixels and (n2xe2x88x92m) white pixels out of n2 pixel arrangements, there are n2Cm patterns for one tone. By using these redundant pixel arrangement patterns for the same quantization level, other information can be expressed.
When the information is an integer, the number of integers which can be allocated is 4, that is, 2 bits in the case of the quantization level 1. In other words, the number BITm of bits which can be allocated at quantization level m is found by using the following equation:
BITm=[log2(n2Cm)]xe2x80x83xe2x80x83(2)
Therefore, for the case of the quantization level 2, BIT2=[log2(22C2)]=log26=2.585 can be found from Equation (2). The number of bits which can be represented by the entire 4-pixel block is BIT0+BIT1+BIT2+BIT3+BIT4=0+2+2.585+2+0=6.585 bits. In this manner, the position of the black pixels as a piece of information can be represented in redundancy. In other words, there are only 4 levels as visual information, while various variations exist in each quantization level in terms of information redundancy.
Furthermore, an image photographed by an imaging apparatus such as a digital camera can be used by carrying out various kinds of processing thereon. For example, panorama processing by photographing a plurality of images and by composing these photographs together so that a huge visual field or depth which cannot be expressed by a single image can be expressed by using a sophisticated image composing technique which is characteristic and available only for digital data. In such panorama processing, in order to compose the images together after photographing, photographing is carried out by using a tripod for panoramic image photographing so that coordinates of the images to be composed together agree geometrically. Alternatively, photographing is carried out in such a manner that the images to be composed together neither overlap nor have a gap. The images are composed together by carrying out distortion correction and interpolation by using geometric information included in each image.
By embedding such supplementary information described above in original data by using deep layer encryption, the supplementary information is prevented from being altered or lost. However, the amount of data to be embedded and quality degradation of original data are in a trade-off relation. For example, when the original data are image data, image quality is degraded if the amount of the embedded information is large. Therefore, in order to retain the quality of the original data, it is necessary to restrict the amount of embedded information.
In the case where image composition is carried out as has been described above, in order to cause the coordinates of the images to be composed together to agree geometrically, a tripod having high rigidity and accuracy is necessary, and photographing is carried out on a large scale. Meanwhile, in the case where the images are composed together by interpolation or the like using geometric information included in the images, no large-scale apparatus is necessary. However, matching up images which has not been carried out upon photographing needs to be carried out at the time of image composition. For example, as shown in FIG. 17, processing to cause features in the images to become consecutive (a manual processing or an advanced algorithm for searching for a characteristic point from the images) needs to be carried out. As shown in FIG. 17, when a lighting condition (in this case, the sun) differs among the images, the state wherein images are formed, such as exposure or fog, is different for each image. Therefore, at a joint of 2 images, density needs to change smoothly. However, when the state of the image is too different between the 2 images, the density cannot be corrected in some cases. As shown in FIG. 18, in the case where a photographing location or magnification is different between the images, the images need to be reduced or enlarged so that characteristic points in each image correspond to each other, and the relation between the images needs to be changed substantially.
Furthermore, depending on aberration of lenses used in digital cameras, as shown in FIG. 19, there are cases of an image being deformed in a shape of an arc (y=fxc2x7xcex8, y=fxc2x7sin xcex8, as shown in FIG. 19(b)) or an image which needs deformation upon image composition (y=fxc2x7tan xcex8, as shown in FIG. 19(a)), and image composition processing becomes more complicated due to such deformation.
In order to cope with this problem, there is a method of attaching photographing information such as a photographing position of an image to image data representing the image as supplementary information. However, in the case where image composition wherein only a portion of an image is used for composition or for geometric distortion is carried out, the supplementary information attached to the image alone may not have enough information in some cases and precise composition is not carried out.
Furthermore, when image processing such as frequency processing is carried out on an image, it is possible to attach frequency information of the image as supplementary information to image data representing the image. However, depending on a characteristic of an imaging apparatus by which the image data have been obtained, the frequency characteristic may be different in each portion of the image. Therefore, even when the frequency information attached to the image is appropriate for a portion of the image, the information may not be appropriate for other portions of the image. Consequently, even if image processing is carried out on a portion of the image according to the supplementary information, appropriate frequency processing may not be carried out on that portion.
The present invention has been created based on consideration of the above problems. An object of the present invention is to provide a method and apparatus for embedding information in original data by using a deep layer encryption method without degrading the quality of the original data and without losing or altering the supplementary information described above, to provide a supplementary information reading method and apparatus for obtaining the supplementary information related to the original data based on the embedded information, and to provide a computer-readable recording medium storing a program to cause a computer to execute the methods.
Another object of the present invention is to provide an information embedding method and apparatus for obtaining supplementary information enabling appropriate processing on a portion of an image even when only the portion is used for composition or processing, and a computer-readable recording medium storing a program to cause a computer to execute the method.
Still another object of the present invention is to provide an image processing method and apparatus for effectively carrying out processing in accordance with the supplementary information on the original data by using a deep layer encryption method, without losing or altering the supplementary information.
A first information embedding method of the present invention is a method of embedding information in original data, and comprises the steps of:
storing supplementary information related to the original image in predetermined storage;
generating storage management information indicating where the supplementary information has been stored; and
embedding the storage management information in the original data.
The xe2x80x9csupplementary informationxe2x80x9d herein referred to means information related to the original data, and copyright information and authentication information of the original data, a photographing condition of an image in the case of the original data being image data, a reading condition in the case of image data read from a film or a print, search information of the original data, and the like can be listed as the supplementary information.
As the xe2x80x9cpredetermined storagexe2x80x9d, a database owned by a user, a database on a network, and the like can be used.
As the xe2x80x9cstorage management informationxe2x80x9d, information regarding the address, a telephone number, and an E-mail address of a person storing the supplementary information, or a URL address on the Internet can be listed, for example.
It is preferable for the first information embedding method of the present invention to embed the storage management information in the original data after carrying out deep layer encryption on the supplementary information.
xe2x80x9cTo embed the supplementary information after carrying out deep layer encryptionxe2x80x9d means to embed the encrypted supplementary information in a redundant portion of the original data, and the method regarding this embodiment is described in the reference by Matsui cited above.
A first information embedding apparatus of the present invention is an apparatus for embedding information in original data, and comprises:
supplementary information storing means for storing supplementary information related to the original image in predetermined storage;
storage management information generating means for generating storage management information indicating where the supplementary information has been stored; and
embedding means for embedding the storage management information in the original data.
In the first information embedding apparatus of the present invention, it is preferable for the embedding means to embed the storage management information in the original data by carrying out deep layer encryption on the supplementary information.
A supplementary information reading method of the present invention comprises the steps of:
obtaining the original data in which the storage management information has been embedded according to the first information embedding method of the present invention;
reading the storage management information from the original data; and
obtaining the supplementary information based on the storage management information.
A supplementary information reading apparatus of the present invention comprises:
data obtaining means for obtaining the original data in which the storage management information has been embedded by the first information embedding apparatus of the present invention;
storage management information reading means for reading the storage management information from the original data; and
supplementary information obtaining means for obtaining the supplementary information based on the storage management information.
A second information embedding method of the present invention comprises the steps of;
dividing an original image represented by original image data into a plurality of areas and for obtaining image data for each area; and
embedding supplementary information related to each area in the image data for each area by carrying out deep layer encryption on the supplementary information.
It is preferable for the second information embedding method to divide the original image into a plurality of block areas.
Alternatively, the original image may be divided into areas of each object included in the original image.
xe2x80x9cTo be divided into areas of each objectxe2x80x9d means to be divided into an area of a figure and an area for its background, when the original image includes a figure and the background, for example.
It is preferable for the supplementary information for each area to be information regarding image processing for the area.
The xe2x80x9cinformation regarding image processingxe2x80x9d means information such as subject coordinate information found from a location and a direction of photographing, presence or absence of a light source such as the Sun, information regarding composition processing such as lens aberration or the like, and information regarding frequency processing such as a frequency characteristic of the image.
It is preferable for the supplementary information for each area to be information representing a keyword for searching for the area.
A second information embedding apparatus of the present invention comprises:
division means for dividing an original image represented by original image data into a plurality of areas and for obtaining image data for each area; and
embedding means for embedding supplementary information for each area in the image data for each area by carrying out deep layer encryption on the supplementary information.
An image processing method of the present invention comprises the steps of:
reading supplementary information related to original data from the original image data such as audio, image, and movie data in which the supplementary information has been embedded by deep layer encryption;
determining processing to be carried out on the original data, based on the supplementary information; and
carrying out processing on the original data, based on the determined processing.
It is preferable for the original data and the supplementary information to be image data acquired by an imaging apparatus and information indicating a photographing condition at the time of acquisition of the original data, respectively.
The xe2x80x9cphotographing conditionxe2x80x9d herein referred to means not only information regarding a characteristic of the imaging apparatus used for photographing, a photographing environment, a purpose of photographing, and a subject, but also information regarding a location, a direction, and the time and date of photographing, for example.
It is preferable for the original data to be image data obtained by reading an image recorded on a film or a print, and, in this case, it is also preferable for the supplementary information to be a reading condition used upon reading the original data.
The xe2x80x9creading conditionxe2x80x9d means a device condition, a characteristic condition, and a set-up condition of a reading apparatus to read the original data.
It is also preferable for the original data and the supplementary information to be image data and process management information regarding the original data, respectively.
The xe2x80x9cprocess management informationxe2x80x9d means information such as history of image processing carried out on the original data, the degree of completeness thereof, the time and date of the processing, a deadline for the processing, and the relationships of jobs.
It is preferable for the supplementary information to be search information showing a search keyword of the original data.
The xe2x80x9csearch informationxe2x80x9d means information which can be a keyword for a search, such as the file name of the original data, and the content of the original data.
It is also preferable for the original data and the supplementary information to be image data and information representing an image related to the original data, respectively.
The xe2x80x9cinformation representing an image related to the original dataxe2x80x9d means, in the case where images of a construction site are the original data, and the name of the site, the time and date of photographing, progress and the like are photographed at the same time, for example, information representing an image obtained by photographing a board describing the name of the site and the like or a memorandum showing the content of the original data.
It is preferable for the original data and the supplementary information to be image data and a processing condition used at the time of output processing on the original data.
The xe2x80x9cprocessing condition used at the time of output processingxe2x80x9d means a printing condition when the original data are printed, for example.
An image processing apparatus of the present invention comprises:
reading means for reading supplementary information from original data in which the supplementary information related to the original data has been embedded by deep layer encryption;
determination means for determining processing to be carried out on the original data, based on the supplementary information; and
processing means for carrying out processing on the original data, based on the determined processing.
An imaging apparatus of the present invention is an imaging apparatus for obtaining image data representing a subject by photographing the subject, and comprises:
embedding means for embedding second image data obtained by photographing a second subject related to a first subject in first image data obtained by photographing the first subject, by carrying out deep layer encryption on the second image data.
It is preferable for the imaging apparatus of the present invention to further comprise selection means for selecting whether or not the second image data are embedded in the first image data.
The first and the second information embedding methods and the image processing method of the present invention may be provided as a program to cause a computer to execute the methods stored in a computer-readable recording medium.
The first information embedding method and apparatus of present invention stores the supplementary information in predetermined storage and the storage management information indicating where the supplementary information has been stored is embedded in the original data. A user who has received the original data in which the storage management information has been embedded reads the storage management information from the original data to obtain the information regarding the storage, and obtains the supplementary information from the storage based on the storage management information.
The storage management information has a comparatively small amount of data such as the address and an E-mail address of a person storing the supplementary information and a URL address on the Internet. Therefore, the storage management information can be embedded in the original data without degrading the quality of the original data. Furthermore, since the storage management information is embedded in the original data unlike information recorded in a header, the information is not lost or altered if various kinds of processing, saving or reading are repeatedly carried out on the original data. Moreover, correspondence between the storage management information and the original data is not confusing. Therefore, the storage management information always follows the original data, and by using the storage management information, the supplementary information is obtained without losing the correspondence with the original data.
Moreover, by embedding the storage management information in the original data after deep layer encryption thereof, accesses to the supplementary information can be limited, which leads to retaining of secrecy of the supplementary information.
According to the second information embedding method and apparatus of the present invention, an original image is divided into a plurality of areas and the image data for each area are obtained. The supplementary information for each area is embedded in the image data for the corresponding area after deep layer encryption of the supplementary information. Therefore, the supplementary information regarding each area, such as search processing and image processing like composition processing and frequency processing for each area, can be embedded. By reading the embedded supplementary information, optimal processing for each area can be carried out based on the supplementary information for each area.
In the case where the supplementary information is embedded in the entire original image data and processing is carried out by dividing the original image, the supplementary information may be lost or altered in some cases. However, according to the present invention, since the supplementary information for each area is embedded in each area, the supplementary information of the divided original image is not lost.
According to the image processing method and apparatus of the present invention, the supplementary information is embedded in the original data by deep layer encryption of the supplementary information. Therefore, unlike information recorded in a header, the supplementary information is not lost or altered if various kinds of processing, saving, or reading are repeatedly carried out on the original data. Furthermore, correspondence between the supplementary information and the original data is not confusing. Therefore, the storage management information always follows the original data and by using the storage management information, appropriate processing can be carried out on the original data.